WHAT IS CLAIMED IS: 



1 . A system for animating a face, comprising: 

a facial reconstruction arrangement for acquiring a base surface model representing a 
three-dimensional shape of a subject's face, and for acquiring a set of displacement fields 
representing motion patterns of the subject's face, wherein the base surface model includes a 
set of control points and each displacement field generates a three-dimensional displacement 
vector that varies over the control points and over an intensity variable; 

an intensity generator to generate a current intensity value for each displacement 
field in the set of displacement fields; 

a deformation unit to combine the displacement vectors generated by the 
displacement fields at the control points and at the current intensity values with the positions 
of the control points to generate a deformed surface model; 

a rendering unit to translate the deformed surface model into a two-dimensional 
image of the face; and 

a video output subsystem to at least one of display and store the two-dimensional 

image. 

2. The system of claim 1, wherein the base surface model represents a shape of the face in a 
neutral, relaxed pose. 

3. The system of claim 1, wherein the facial reconstruction arrangement includes: 

a surface acquisition arrangement for acquiring a set of surface models that 
approximate shapes of the subject's face; and 

a storage arrangement for storing the set of acquired surface models. 

4. The system of claim 3, wherein each of the surface models of the set of surface models 
acquired from the subject's face includes: 

a set of three-dimensional points measured from the subject's face; 

a topological model representing a set of vertices and connections between them; 

and 
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an association between the set of three-dimensional points and the vertices which 
determines an embedding of the topological model in three-dimensional space. 

5. The system of claim 3, wherein the surface acquisition arrangement includes an active 
sensing arrangement for surface measurement. 

6. The system of claim 5, wherein the active sensing arrangement projects a grid pattern for 
surface measurement. 

7. The system of claim 5, wherein the active sensing arrangement projects a pattern of 
multiple parallel stripes for surface measurement. 

8. The system of claim 3, wherein the set of acquired surface models includes the base 
surface model. 

9. The system of claim 3, wherein the surface acquisition arrangement acquires a sequence 
of surface models from a sequence of facial poses involving a particular muscle movement; 
and wherein the facial reconstruction arrangement is configured to associate a sequence of 
increasing intensity values with the sequence of surface models, the intensity values 
representing degrees of muscle contraction for the particular muscle movement in 
corresponding poses in the sequence of facial poses. 

10. The system of claim 9, wherein the facial reconstruction arrangement further includes a 
jaw immobilizer to minimize differences in jaw position during different poses in the 
sequence of facial poses involving the particular muscle movement. 

11. The system of claim 9, wherein the facial reconstruction arrangement further includes a 
a surface registration unit to geometrically align the surface models to minimize differences 
in head position. 
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12. The system of claim 11, wherein the surface registration unit applies an iterative closest 
point technique to geometrically align the surface models in the sequence of surface models. 

13. The system of claim 9, wherein the facial reconstruction arrangement further includes a 
displacement field derivation unit to derive one of the displacement fields in the set of 
displacement fields from the sequence of surface models. 

14. The system of claim 13, wherein the displacement field derivation unit is configured to: 

re-position the control points of the base surface model to fit the shape of each 
particular surface model in the sequence of surface models to produce a sequence of 
deformed surface models approximating the original sequence of surface models but having 
the control points of the base surface model; 

calculate the displacements of the control points of the base surface model at each 
particular intensity value in the sequence of intensity values, by calculating the displacement 
of each control point from its position in the deformed surface model associated with the 
first intensity value in the sequence of intensity values to a position in the deformed surface 
model associated with the particular intensity value; and 

derive the displacement field by determining the displacements of the control points 
of the base surface model at intensity values intermediate to the intensity values in the 
sequence of intensity values, by interpolating between the displacements at the intensity 
values in the sequence of intensity values. 

15. The system of claim 14, wherein the displacement field derivation unit performs a linear 
interpolation of the displacements at the intensity values in the sequence of intensity values. 

16. The system of claim 14, wherein the displacement field derivation unit performs a non- 
linear interpolation of the displacements at the intensity values in the sequence of intensity 
values. 

17. The system of claim 14, wherein the displacement field derivation unit maps the control 
points to positions in the surface of the particular surface model. 
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18. The system of claim 14, wherein the displacement field derivation unit maps regions of 
the surface of the base surface model to corresponding regions of the surface of the 
particular surface model. 

19. The system of claim 18, wherein the surface acquisition arrangement acquires a 
photographic image of the subject's face at the same time as it performs the three- 
dimensional measurements used to make a surface model in the set of surface models, and 
wherein the surface acquisition arrangement associates the photographic image with the 
surface model as a texture, and wherein the displacement field derivation unit maps regions 
of the surface of a surface model to corresponding regions of the surface of another surface 
model using the textures of the two surface models. 

20. The system of claim 19, wherein the displacement field derivation unit uses a network of 
lines that are drawn on the subject's face and appear in the texture of each surface model in 
the set of surface models to map regions of the surface of a surface model to corresponding 
regions of a surface of another surface model. 

21. The system of claim 1, wherein the current intensity values for the displacement fields 
are sampled from a set of time-varying functions. 

22. The system of claim 1, wherein the current intensity values for the displacement fields 
are manually input. 

23. The system of claim 1, wherein the current intensity values for the displacement fields 
are supplied by a speech animation program. 

24. The system of claim 1, wherein the deformation unit combines the displacement vectors 
generated by the displacement fields at each control point with the position of the control 
point by vector sum. 
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25. The system of claim 1, wherein the rendering unit translates the deformed surface model 
into the two-dimensional image using color values associated with the base surface model. 

26. The system of claim 1 , wherein the two-dimensional image includes a bit map. 

27. A method for animating a face, comprising: 

acquiring a base surface model representing a three-dimensional shape of the face 
and including a set of control points; 

acquiring through three-dimensional measurement a set of displacement fields 
approximating motion patterns of a subject's face, each displacement field generating a 
three-dimensional displacement vector varying over the control points of the base surface 
model and over an intensity variable; 

generating a current intensity value for each displacement field in the set of 
displacement fields; 

combining the displacement vectors generated by the displacement fields at the 
control points and at the current intensity values with the positions of the control points to 
generate a deformed surface model; 

rendering the deformed surface model to generate a two-dimensional visual image of 
the face; and 

displaying or storing the two-dimensional image. 

28. The method of claim 27, wherein the base surface model represents a shape of the face 
in a neutral, relaxed pose. 

29. The method of claim 27, wherein the step of acquiring the set of displacement fields 
includes acquiring a set of surface models approximating shapes of the subject's face. 

30. The method of claim 29, wherein the set of surface models acquired from the subject's 
face includes the base surface model. 
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3 1 . The method of claim 29, wherein each of the surface models acquired from the subject's 
face includes: 

a set of three-dimensional points measured from the subject's face; 

a topological model representing a set of vertices and connections between them; 

and 

an association between the set of three-dimensional points and the vertices which 
determines an embedding of the topological model in three-dimensional space. 

32. The method of claim 29, wherein the set of surface models is acquired from the 
subject's face using measurement by active sensing. 

33. The method of claim 32, wherein the active sensing includes projecting a grid pattern. 

34. The method of claim 32, wherein the active sensing includes projecting a pattern of 
multiple parallel stripes. 

35. The method of claim 29, wherein the set of acquired surface models includes a sequence 
of surface models acquired from a sequence of facial poses involving a particular muscle 
movement; and wherein a sequence of increasing intensity values is associated with this 
sequence of surface models, the intensity values representing degrees of muscle contraction 
for the muscle movement in the corresponding poses in the sequence of facial poses. 

36. The method of claim 35, wherein the subject's jaw is immobilized to minimize 
differences in jaw position during different poses in the sequence of facial poses of the 
particular muscle movement. 

37. The method of claim 35, wherein the surface models in the sequence of surface models 
are geometrically aligned to minimize differences in head position. 

38. The method of claim 37, wherein the geometric alignment of the surface models 
includes application of an iterative closest point technique. 
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39. The method of claim 35, wherein one of the displacement fields in the set of 
displacement fields is derived from the sequence of surface models. 

40. The method of claim 39, wherein the derivation of the displacement field comprises the 
steps of: 

re-positioning control points of the base surface model to fit the shape of each 
particular surface model in the sequence of surface models to produce a sequence of 
deformed surface models approximating the original sequence of surface models but having 
the control points of the base surface model; 

calculating the displacements of the control points of the base surface model at each 
particular intensity value in the sequence of intensity values, by calculating the displacement 
of each control point from its position in the deformed surface model associated with the 
first intensity value in the sequence of intensity values to its position in the deformed 
surface model associated with the particular intensity value; and 

deriving the displacement field by determining the displacements of the control 
points of the base surface model at intensity values intermediate to the intensity values in 
the sequence of intensity values, by interpolating between the displacements at the intensity 
values in the sequence of intensity values. 

41 . The method of claim 40, wherein the step of interpolating the displacements at the 
intensity values in the sequence of intensity values includes performing a linear 
interpolation. 

42. The method of claim 40, wherein the step of interpolating the displacements at the 
intensity values in the sequence of intensity values includes performing a non-linear 
interpolation. 

43. The method of claim 40, wherein the step of re-positioning the control points of the base 
surface model to fit the shape of the particular surface model includes mapping the control 
points to positions in the surface of the particular surface model. 
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44. The method of claim 43, wherein the mapping of the control points of the base surface 
model to positions in the surface of the particular surface model includes mapping regions 
of the surface of the base surface model to corresponding regions of the surface of the 
particular surface model. 

45. The method of claim 44, wherein each surface model in the set of surface models has 
associated with it a photographic image of the subject's face that is acquired by the surface 
acquisition system at the same time as the three-dimensional measurements used to make 
the surface model, the photographic image being mapped to the surface of the surface model 
as a texture; and wherein the mapping of regions of the surface of the base surface model to 
corresponding regions of the particular surface model includes usage of the textures of the 
two surface models. 

46. The method of claim 45, wherein prior to acquiring the set of surface models a network 
of lines is drawn on the subject's face dividing it into regions, the network of lines 
consequently appearing in the texture of each surface model; and wherein the mapping of 
regions of the surface of the base surface model to corresponding regions of the particular 
surface model includes usage of the network of lines appearing in the textures of the two 
surface models. 

47. The method of claim 27, wherein the current intensity values for the displacement fields 
are sampled from a set of time-varying functions. 

48. The method of claim 27, wherein the current intensity values for the displacement fields 
are manually input. 

49. The method of claim 27, wherein the current intensity values for the displacement fields 
are supplied by a speech animation program. 

50. The method of claim 27, wherein the step of combining the displacement vectors 
generated by the displacement fields at each control point with the position of the control 
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point includes calculating a vector sum. 



51. The method of claim 27, wherein the rendering step includes using color values 
associated with the base surface model. 

52. The method of claim 27, wherein the two-dimensional image includes a bit map. 

53. An animation system, comprising: 

an arrangement for acquiring by three-dimensional measurement a surface model 
representing at least a portion of an object and a set of displacement fields representing 
motion patterns associated with the object; 

an intensity generator to generate an intensity value for each displacement field in 
the set of displacement fields; 

a deformation unit to generate a deformed surface model by applying the set of 
displacement fields to the surface model using the intensity values; and 

a rendering unit to translate the deformed surface model into a two-dimensional 

image. 
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